Introduction

In an interview that immediately became a cult, President Trump stated that United States have “the number one low mortality rate” for COVID-19. The CNN reacted to this interview with a piece noting that the US has more deaths per 100,000 people than many other countries.Footnote 1 Leaving asides Trumpian overstatements—the US does not have the number one low mortality rate by any metric—this exchange captures how looking at different ways to measure COVID-19 deaths can feed into different narratives. The data used by Trump was actually on the case fatality rate (CFR), which is the ratio between the number of deaths divided by the number of diagnosed cases. At the moment of the interview, the CFR of the United States was relatively low because the US had just started diagnosing a large number of new cases, which increased the value of the denominator. CNN responded with a measure of deaths per capita that is instead connected with the case mortality rate (CMR), which is the number of deaths divided by a total number of people. But what is the impact on people’s attitudes, policy preferences and behaviours of these different ways to convey information about the mortality of COVID-19? This letter answers this question.

While the mortality of a disease can be measured in many ways, there are three indicators that are frequently used.Footnote 2 First, one can look at the CFR, which expresses the percentage of people that have been diagnosed with a disease that die from it. Second, one can look at the Infection Fatality Rate (IFR), which captures the risk of dying from a given disease if you contract it. It is calculated by dividing the number of deaths caused by a given disease by the number of people that contract the disease. Notably, in the case of COVID-19 and of other diseases it is impossible to know with precision the denominator of the IFR because some people that contract the disease will not be diagnosed. Consequently, there is always some uncertainty surrounding the true value of the IFR.Footnote 3. Third, one can look at the crude mortality rate, which captures the probability that any individual in the population will die from the disease. Consequently, it is a ratio of the number of people that die from a disease divided by the total number of individuals in a population. In this letter, we test whether the choice of the rate affects people’s attitudes, policy preferences and behaviours. Moreover, we study whether people understand the meaning of these different ratios.

Our work relates with several strands of literature. To begin with, it relates with the recent works showing that the way in which COVID-19-related information is conveyed affects people’s attitudes, policy preferences and behaviours (Sotis et al., 2020; Garfin et al., 2020). On this subject, it is relevant to refer to the recent large-scale international analysis by Van Bavel et al. (2020) that shows how support for public health behaviours is generally more significant where individuals identify more strongly with their nation. Arora and Grey (2021) provide a useful review of the extant studies on how COVID-19 impacts behaviour both short and long term, while Kooistra and van Rooij (2020) illustrate behavioural findings from 45 studies on pandemic compliance, with a focus on social distancing.

Secondly, our work relates with the literature on health literacy, which is defined by the Center for Disease Control and Prevention (CDC) as “the degree to which an individual has the capacity to obtain, communicate, process, and understand basic health information and services to make appropriate health decisions”.Footnote 4 The relationship between health status and poor literacy is also highlighted in the analysis by Nutbeam (2008) who distinguishes health literacy as a clinical risk and personal asset. Low health literacy in the US has also been demonstrated in the study of Paasche-Orlow et al. (2005). There is ample evidence that a good level of health literacy is an important factor in facilitating the prevention of non-communicable diseases, and the COVID-19 pandemic showed that it is key also to prevent the spread of communicable diseases and to aid people in complex everyday decision making in the midst of a pandemic (Paakkari and Okan, 2020).

Last, our work relates to the literature on how the way in which a message is conveyed affects health-related choices. Many studies show that the choice of words has an important impact and, therefore, it is possible that the different usage of COVID-19 indicators affects individual choices. For instance, Ajzenman et al. (2020) showed how compliance to preventive measures such as social distancing significantly dropped in Brazil following president Bolsonaro’s ‘empathetic’ dismissal of the risks associated with COVID-19. This is also in line with the many analyses that show mistrust in scientific guidelines (Plohl and Musil, 2020). The critical care capacity of the US (Moghadas et al., 2020) justifies the need to understand what COVID-19 indicator is more effective in favouring containment measures against the virus in an epoch where individuals are asked to dramatically change their behaviour (Van Bavel, 2020).

Experimental design

We devised a double-blind experiment approved by the Nanjing Audit University IRB to test how the use of different mortality rates affects people’s attitudes and policy preferences. Moreover, we study if people understand the mortality rates generally used in the media. We recruited a sample of n = 1942 U.S. residents on Amazon Mechanical Turk (MTurk) and excluded non-eligible individuals; namely, individuals younger than 18 and non-U.S residents. Individuals were paid $0.30 to answer the questions proposed in the experiment conducted on the Qualtrics platform. In a context where participants are forced to answer a specific question in order to proceed to the next one, some participants may feel the urge to skip said question and pick a random answer in order to proceed. We opt not to force them to answer any question to avoid wrongfully picked answers that could bias our results. This is in line with the analysis by Freira et al. (2015) on the trade-off between quality of data and forced answers. One limitation of this design is the lack of attention checks. Our decision, however, stems from recent evidence shown by Qualtrics expert Vannette (2017) that these may harm data quality.

With respect to individuals’ characteristics, we observe that, on average, individuals in the sample are 38.29 years old, with a standard deviation of 12.26. Less than half of our samples are female (41.95%). The distribution of income levels varies across the sample, with 2.72% having an income of more than $150,000 or more, 7.56% of them earning $100,000–$149,999, and 16.76% declaring they have an income of $50,000–$59,999. With respect to education, we observe that most individuals in our sample possess a bachelor’s degree (56.36%), followed by individuals with a master’s degree (20.29%) and doctoral degree (1.03%). Some differences are observed also in relationship to political preferences; namely, 1145 and 800 consider themselves as, respectively, democrats and republicans. Finally, 41%, 21%, and 15% of the individuals report reading news related to COVID-19 once, twice, and three or more a day (Table 1). The sample is balanced with respect to the three treatment groups and the control group (see Table 2), except for income levels, somewhat higher in the groups treated with information about the crude mortality and infection fatality rate; and the slightly higher density of individuals who expressed republican viewpoints in the crude mortality rate group.

Table 1 Descriptive statistics.
Full size table
Table 2 Descriptive statistics by treatment group.
Full size table

All participants were shown a screen with sufficient data to calculate all the three mortality rates considered in this experiment. The data used to calculate the different mortality rates was taken from the website www.worldometers.info. To calculate the IFR we needed an estimate of the number of people that have been infected with COVID-19 but have not been diagnosed. For this purpose, we rely on the updated estimates of the US Centers for Disease Control and Prevention.

In the treatment stage, one-quarter of the respondents were randomly assigned to the Infection Fatality Rate Group (IFR Group), one quarter to the Case Fatality Rate Group (CFR Group) and one quarter to Crude Mortality Rate Group (CMR Group). The last quarter of respondents was assigned to the Control Group (Control), who were not provided with any COVID-19 indicator. We asked respondents four sets of questions: (i) attitudes and policy preferences, (ii) behavioural response, (iii) understanding of COVID-19 indicators, and (iv) standard demographic questions. In the supplementary material, we report the questions we asked and the order in which they were asked.

Attitudes and policy preferences

To begin with, we asked respondents the following questions to assess their level of support for some widely debated COVID-19 related policies: (i) Would you support a law that mandates the use of protective masks during the pandemic?, (ii) Would you support a law that bans large events indoor with many people during the pandemic?, (iii) Would you support a law that bans indoor religious events during the pandemic?, (iv) It has been suggested that the US is not testing enough people for COVID-19 and that this contributes to the diffusion of the virus. Would you support a tax to finance an increase in testing capacity?. All these questions were asked using five points Likert scale from “strongly oppose” to “strongly support”. We then asked respondents how worried they are about the health and the economic crises caused by the COVID-19. These questions were asked using five points Likert scale from “not worried at all” to “extremely worried”.

Behaviours

Afterwards, we asked respondents about how they planned to behave outside of the workplace in the 2 weeks following the survey. After asking them whether they intended to socialise in small gatherings and in large gatherings, we then asked them if they planned to see older friends or people with pre-existing conditions if they planned to stay home unless they needed to shop for groceries or essential items if they planned to wear a mask when going out and if they intended to disinfect the items bought at the supermarket. These questions were asked using a zero to ten scale, where zero means individuals plan to definitely not engage in the behaviour and ten means individuals plan to engage in the behaviour with absolute certainty. Lastly, we asked two questions on how their behaviour changed in comparison to the pre-pandemic period. In particular, we asked them if they attended more or less small and large gatherings in comparison to the pre-pandemic period. This question was asked using a ten points scale, from −5 to 5, where −5 means that they will engage in the activity much less and +5 means they will engage in the activity much more.

Understanding

We then tested respondents’ understanding of the mortality rates. To increase external validity and to avoid priming respondents, we asked attitudes and policy preferences before testing their understanding of COVID-19 indicators. This allows obtaining respondents’ policy preferences, without having participants think thoroughly about the mortality rates and their meaning in a way that they would be unlikely to do when reading actual news.

We test understanding of mortality rates recurring to three questions. Firstly, we provide participants with data for Singapore that are necessary to calculate all three mortality rates. We then ask participants to calculate the mortality rate to which they were assigned. In order to answer correctly to the question, they have to identify the correct numbers and carry out the appropriate division. This allows them to identify the correct answer among the four we present to them. For instance, the respondents in the CMR Group were asked to calculate the CMR of COVID-19 based on the data we provided them with.

Secondly, we ask respondents to identify the right definition of the mortality rate to which they were assigned among the definitions we provided them with. Lastly, we ask respondents to calculate the mortality rate to which they were assigned with respect to an imaginary disease. The structure of the question is identical to that of the first understanding question. However, the data now refers to an imaginary disease instead of being related to COVID-19.

We concluded by collecting standard demographic information about the respondents.

Results

Results from an ordinal logit modelFootnote 5 show that depending on the type of COVID-19 indicator individuals are presented with, individual behaviour, policy preferences, and the level of understanding of such indicators change significantly.

$${\mathrm {Outcom{e}}}_{i}={\mathrm {treatmen{t}}}_{t}+{\gamma }_{c,i}+{{{\Omega }}}_{\mathrm {{f}}}+\epsilon$$

where Outcomei is the set of outcome variables; treatmentt is a dummy equal to 1 if the subject is in the treatment group t; γi is the control c fore subject i; Ωf is the set of fixed effects such as age.

With respect to attitudes and policy preferences (Table 3), we observe that compared to a control group that is not provided with any COVID-19 indicator, individuals are 26.9 percentage points more likely to support a law that enforces the use of masks if they are presented with the crude mortality rate (CMR). When ignoring controls, informing individuals of the COVID-19 CFRs decreases the support for a law that bans indoor events by 24.7 percentage points. Due to the lack of significance in the model when controls are added, we believe results could be partially driven by some omitted variables. For instance, we find that the effect of CFR on banning indoor events remains significant when the news-related controls are not taken into account. This means that the result is driven by whether people read COVID-19 news or not. For those who do not read such information, the effect of CFR is not statistically significantly different from zero. Taking other demographic controls does not make the result insignificant. This is in line with the argument by Duenas et al. (2021) that socioeconomic conditions influence people’s behaviour during the COVID-19 pandemic. They observed that for individuals with worse socioeconomic conditions the gap in mobility flows during lockdown was small compared to pre-pandemic times. While individuals with higher income may be able to stay at home for longer, poverty will not always allow it.

Table 3 COVID-19 mortality rates and policy preferences.
Full size table

The usage of COVID-19 crude mortality rate to inform individuals about the seriousness of the pandemic is once again the strongest treatment with respect to future worries (Table 4). Compared to the control group and controlling for individuals fixed effects, individuals who are treated with information about the COVID-19 crude mortality rate are 28.2 percentage points more worried about a potential economic crisis. No treatment affected people’s worries with regards to the prominent health crisis.

Table 4 COVID-19 mortality rates and health and economic worry.
Full size table

With respect to behaviour (Tables 5 and 6), we observe that presenting individuals with the infection fatality rate (IFR) is the most negative effective indicator in changing behaviour. Compared to the control group, individuals treated with such information are 23.9 percentage points less likely to stay home unless it is essential for work, medical treatment, or groceries. Similarly, they are 22.5 percentage points less likely to wear a mask together with individuals treated with the COVID-19 CFR, who are 24.2 percentage points less likely to do so. With respect to disinfecting products bought at the supermarket, findings are only significant at the 10% level. However, they still confirm the negative influence of the infection fatality rate (−19.4), possibly underestimated by individuals who associate it to a mere infection rate, and of CFR (−20.6), used less frequently by media.

Table 5 COVID-19 mortality rates and behaviour (gatherings).
Full size table
Table 6 COVID-19 mortality rates and behaviour (other).
Full size table

Finally, we also investigated the extent to which individuals understand the meaning of the COVID-19 indicators used daily by media (Table 7). Each group was provided with the definition of the COVID-19 rate assigned to their treatment. Results show that when showing them the exact same text originally provided for the US this time applied to Singapore, individuals treated with information on the COVID-19 infection fatality rate were 28.9 percentage points less likely to understand what the indicator meant, compared to individuals treated with information on the COVID-19 CFR. Such individuals were also 57.9 percentage points less likely to identify the right definition of the COVID-19 indicator they had been assigned to—in this case the infection fatality rate—right after they had been provided with such definition earlier during the experiment. When providing them with a scenario about an imaginary disease, individuals presented with the COVID-19 crude mortality rate were 87.3 percentage points more likely to choose the wrong answer.

Table 7 Understanding COVID-19 Indicators.
Full size table

Discussion

In this article, we showed that individuals are less likely to behave efficiently during a pandemic when presented with the infection fatality rate. This could be attributed to the fact that individuals associate the name of such a rate with the closest mild disease they are accustomed to; namely, a flu. Being informed about the infection fatality rate of COVID-19 significantly reduces lock-down behavioural decisions. This could possibly be attributed to the difficulty experienced by individuals to understand how IFR is computed. Its unclear and non-ecological nature also make it the least likely indicator used by the media. Luo et al. (2021) recently showed how even survey results often underestimate the true IFR. In line with this, Staerk et al. (2021) argue that the true fatality rate is ‘hard to calculate’. This is also due to the fact that IFR estimates change quite quickly (Meyerowitz-Katz and Merone, 2020). On the other hand, using crude mortality rate appears to drive policy preferences that help stop the spread of the virus, as well as boost the level of concern with respect to a potential economic crisis. This could potentially be explained by the richer piece of information provided to individuals exposed to CMR; namely, the value of the entire population. This allows individuals to better understand how many economically active individuals were infected or diagnosed with the virus.

In line with Rice and Cooper (1954), the crude mortality rate can be used to estimate the impact of the pandemic on economically relevant factors such as countries’ GDP. They claimed that ‘for public programs, such as the control and eradication of disease, [...] the valuation of human lives is a basic requirement for the proper calculation of the benefits to be derived’. Clearly, if a person is sick, the economic value she can produce for herself and the more extended society decreases or is even nullified. On this subject, Conley (1976), Weinstein (1980), and Viscusi (2008) showed how the economic value of human life is strictly related to the value and demand for safety, mortality probabilities, and small risks of death, of which COVID-19 can be responsible. This also connects to the recent study by Power (2020) on the economic impact produced by quality of life, deeply influenced by aspects of health typical of this pandemic.

Despite the fact that such commonly used COVID-19 indicators can be used as a promising means of information able to influence attitudes positively, our findings show that individuals lack the level of understanding necessary to make moral decisions. Particularly, we note that providing people with information about COVID-19 indicators does increase their approval for preventive policies but does not necessarily encourage engagement in such preventive measures. On the other hand, oftentimes it reduces their willingness to comply with preventive behaviour. Using the crude mortality rate indicator, for instance, incentives attendance of small gatherings. Using the infection fatality rate causes individuals to stay less at home, while the CFR decreases individuals’ propensity to use a mask.

Thus, our findings feed into the extant literature on moral hypocrisy; namely, people wanting to appear moral but not willing to incur the costs of being moral. In this regard, Sulik et al. (2021) recently showed how science boosts approval but not compliance to COVID-19 rules. This, of course, also relates to how people perceive membership in the more extended society. Lower COVID-19-related hypocrisy is found for people who are high on collectivism and global personal impact (Bok et al., 2020). Lower levels of prevention are instead typical of citizens who support populist governments, such as Bolsonaro in Brazil (Ajzenman et al., 2020) and Trump in the US (Allcott, 2020). The fact that more individuals who identify with the political views of the Republicans are present in the CMR group could partially explain why the misunderstanding of COVID-19 preventive rules is so significant. The work on Fox News data by Ash et al. (2020) is relevant in this regard in that it shows how using a narrative that distrusts science and minimises COVID-19 risks significantly impacts behaviour.

In general, the efficacy of COVID-19 indicators could be associated with risk perception, as well as cultural characteristics. As found by Huynh (2020a), in countries where the uncertainty avoidance index, or how well people cope with anxiety, is low, it is more likely for people to comply with preventive measures such as social distancing. Not gathering in public, staying at home, and avoiding going to grocery shops often may help individuals avoid unpredictable situations. This then connects to risk preferences. Another recent study by Huynh (2020b) shows how risk-averse behaviours help to contain the virus by increasing risk perception from public health communications.

Conclusions

Our findings depart from experimental evidence provided by Freira et al. (2021) on the lack of correlation between communication strategies to contain the virus and support for preventive measures. On the other hand, they support the recent analysis by Allcott et al. (2020), according to which beliefs about the risks and severity of the pandemic influence individuals’ behaviour towards policies put in place to contain the virus.

Our experimental evidence could suffer from omitted variable bias, as well as the potential inclusion of non-focused participants. Moreover, data are solely based on the US population, where effects derived from partisanship are renowned to be strong. Thus, further research should be carried out to investigate behavioural compliance to COVID-19 rules for different segments of the population and in different areas of the world. In particular, it would be relevant to understand how the efficacy of health communications regarding COVID-19 changes in growing phases of the pandemic. While individuals may be more prone to comply with safety measures during acute pandemic periods, their optimism and psychological tiredness caused by lockdown measures may imply reception of COVID-19 indicators could be less effective.

As argued by Anderson et al. (2020), policymakers are facing a process of trial and error. While overwhelming individuals with numbers and COVID-19 indicators may have a short-term impact that helps contain the virusFootnote 6, it is evidently necessary to provide individuals with complete information about the indicators they are looking at. Only then, will individuals be able to take morally sound decisions in such a critical time. We, therefore, recommend policy makers to (i) regularly provide clear definitions of the various COVID-19 indicators to compensate for the lack of objective information often provided by the media; (ii) favour the usage of the crude mortality rate during normal pandemic times, when the objective is to make people used to wearing masks indoor and regularly use disinfectants; (iii) encourage the use of the CFR when the aim is to increase people’s understanding of the pandemic’s gravity. This is especially useful when governments seek people’s support of measures that limit the organisation of social events.

Using the right type of information is critical also in regards to vaccination policies. On this subject, Tentori et al. (2021) have recently shown how misunderstanding vaccine efficacy predicts a lower behavioural propensity to get vaccinated. Motta et al. (2021) also found that health communicators that focus on the personal health risks of not getting the jab are effective in increasing compliant behaviour. While during the outbreak of COVID-19 the major issue was to contain the virus, the current issue is to be able to live with it. On the one hand, policymakers need to address the issue of people’s hesitancy by promoting the usage of complete and correct information. On the one hand, they need to ensure absorption of health communications results in actual behaviour, rather than moral hypocrisy.